{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 神经网络常用激活函数\n",
    "### （Sigmod, tanh, ReLU）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. sigmod\n",
    "#### 表达式\n",
    "$$\n",
    "\\begin{align}\n",
    "g\\left( z\\right) & =\\dfrac {1}{1+e^{-z}}\n",
    "\\end{align}\n",
    "$$\n",
    "#### 求导：\n",
    "$$\n",
    "\\begin{align}\n",
    "g\\left( z\\right)^{'} & =\\left( \\dfrac {1}{1+e^{-z}}\\right)^{'} \\\\\n",
    "& = \\dfrac {1^{'}\\left( 1+e^{-z}\\right) -1\\left( 1+e^{-z}\\right)^{'} }{\\left( 1+e^{-z}\\right) ^{2}} \\\\\n",
    "& = \\dfrac {-1\\left( -e^{-z}\\right) }{\\left( 1+e^{-z}\\right) ^{2}} \\\\\n",
    "& = \\dfrac {e^{-z} }{\\left( 1+e^{-z}\\right) ^{2}} \\\\\n",
    "& = \\dfrac {1}{1+e^{-z}}\\dfrac {e^{-z}}{1+e^{-z}} \\\\\n",
    "& = \\dfrac {1}{1+e^{-z}}\\dfrac {1+e^{-z}-1}{1+e^{-z}} \\\\\n",
    "& = \\dfrac {1}{1+e^{-z}}\\left( 1-\\dfrac {1}{1+e^{-z}}\\right) \\\\\n",
    "& = g(z)(1-g(z))\n",
    "\\end{align}\n",
    "$$\n",
    "#### 作用：\n",
    "将输出$z$激活映射到$(0,1)$之间\n",
    "\n",
    "#### 缺点：\n",
    "\n",
    "1. 当$z$非常大或非常小时，sigmod的导数$g(z)^{'}$将接近0，会导致向下层传播或反向传播更新权重$W$时，使$W$需要修改的值（$W$的梯度）非常小（接近0），使梯度更新非常缓慢，即梯度消失\n",
    "2. 函数的输出不是以0为均值，均值是0.5，不便于下层的计算\n",
    "\n",
    "#### 使用：\n",
    "\n",
    "1. 二分类算法（如逻辑回归）的最后（比如让逻辑回归输出值变成$(0, 1)$的值）\n",
    "2. 神经网络的最后一层，不用在隐藏层中，作为输出层做二分类作用\n",
    "\n",
    "\n",
    "\n",
    "# 2. tanh\n",
    "#### 表达式\n",
    "$$\n",
    "g(z) = \\frac{e^z-e^{-z}}{e^z+e^{-z}}\n",
    "$$\n",
    "#### 求导\n",
    "$$\n",
    "\\begin{align}\n",
    "g(z)^{'} & = \\left(\\frac{e^z-e^{-z}}{e^z+e^{-z}} \\right)^{'} \\\\\n",
    "& = …… \\\\\n",
    "& = \\frac{4}{(e^z+e^{-z})^2} \\\\\n",
    "& = …… \\\\\n",
    "& = 1-g(z)^2\n",
    "\\end{align}\n",
    "$$\n",
    "#### 作用：\n",
    "将输出$z$激活映射到$(-1,1)$之间\n",
    "\n",
    "#### 缺点：\n",
    "\n",
    "1. 同sigmod，易梯度消失\n",
    "\n",
    "#### 优点：\n",
    "\n",
    "1. 均值为0，比sigmod好\n",
    "\n",
    "#### 使用：\n",
    "\n",
    "1. 神经网络最后一层，偶尔也能看到用在隐藏层\n",
    "\n",
    "\n",
    "\n",
    "# 3. ReLU\n",
    "#### 表达式\n",
    "$$\n",
    "\\begin{align}\n",
    "g(z) & = \\begin{cases}\n",
    "\tz, & if\\ z>0\\\\\n",
    "\t0, & if\\ z<0\n",
    "\\end{cases}\n",
    "\\end{align}\n",
    "$$\n",
    "#### 求导\n",
    "$$\n",
    "\\begin{align}\n",
    "g(z)^{'} & = \\begin{cases}\n",
    "\t1, & if\\ z>0\\\\\n",
    "\t0, & if\\ z<0\n",
    "\\end{cases}\n",
    "\\end{align}\n",
    "$$\n",
    "#### 缺点：\n",
    "\n",
    "1. 当输入为负时，梯度为0，梯度消失\n",
    "\n",
    "#### 优点：\n",
    "\n",
    "1. 在输入为正数时，不存在梯度消失问题\n",
    "2. 计算速度快，计算公式是线性关系，sigmod和tanh要计算指数，计算慢"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow MNIST"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.datasets import fetch_mldata\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "from sklearn.model_selection import train_test_split\n",
    "import tensorflow as tf\n",
    "import warnings\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data\\train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data\\train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data\\t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data\\t10k-labels-idx1-ubyte.gz\n",
      "step 0,train_accuracy= 0.12\n",
      "step 1000,train_accuracy= 0.94\n",
      "step 2000,train_accuracy= 1\n",
      "step 3000,train_accuracy= 0.98\n",
      "step 4000,train_accuracy= 0.98\n",
      "step 5000,train_accuracy= 1\n",
      "step 6000,train_accuracy= 1\n",
      "step 7000,train_accuracy= 1\n",
      "step 8000,train_accuracy= 0.98\n",
      "step 9000,train_accuracy= 1\n",
      "step 10000,train_accuracy= 1\n",
      "step 11000,train_accuracy= 1\n",
      "step 12000,train_accuracy= 1\n",
      "step 13000,train_accuracy= 0.98\n",
      "step 14000,train_accuracy= 1\n",
      "step 15000,train_accuracy= 1\n",
      "step 16000,train_accuracy= 1\n",
      "step 17000,train_accuracy= 1\n",
      "step 18000,train_accuracy= 1\n",
      "step 19000,train_accuracy= 1\n",
      "test_accuracy= 0.9921\n"
     ]
    }
   ],
   "source": [
    "# 读取数据\n",
    "# mnist = fetch_mldata('MNIST original')\n",
    "# X, y = mnist[\"data\"], mnist[\"target\"]\n",
    "# print(X.shape, Y.shape)\n",
    "mnist = input_data.read_data_sets('MNIST_data', one_hot=True)\n",
    "sess = tf.InteractiveSession()\n",
    "# 构建cnn网络结构\n",
    "# 自定义卷积函数（后面卷积时就不用写太多）\n",
    "def conv2d(x,w):\n",
    "    return tf.nn.conv2d(x, w, strides=[1,1,1,1], padding='SAME') \n",
    "# 自定义池化函数 \n",
    "def max_pool_2x2(x):\n",
    "    return tf.nn.max_pool(x, ksize=[1,2,2,1], strides=[1,2,2,1], padding='SAME')\n",
    "# 设置占位符，尺寸为样本输入和输出的尺寸\n",
    "x = tf.placeholder(tf.float32,[None,784])\n",
    "y_ = tf.placeholder(tf.float32,[None,10])\n",
    "x_img = tf.reshape(x,[-1,28,28,1])\n",
    "\n",
    "# 设置第一个卷积层和池化层\n",
    "w_conv1 = tf.Variable(tf.truncated_normal([3,3,1,32], stddev=0.1))\n",
    "b_conv1 = tf.Variable(tf.constant(0.1, shape=[32]))\n",
    "h_conv1 = tf.nn.relu(conv2d(x_img, w_conv1) + b_conv1)\n",
    "h_pool1 = max_pool_2x2(h_conv1)\n",
    "\n",
    "# 设置第二个卷积层和池化层\n",
    "w_conv2 = tf.Variable(tf.truncated_normal([3,3,32,50], stddev=0.1))\n",
    "b_conv2 = tf.Variable(tf.constant(0.1, shape=[50]))\n",
    "h_conv2 = tf.nn.relu(conv2d(h_pool1, w_conv2) + b_conv2)\n",
    "h_pool2 = max_pool_2x2(h_conv2)\n",
    "\n",
    "# 设置第一个全连接层\n",
    "w_fc1 = tf.Variable(tf.truncated_normal([7*7*50,1024], stddev=0.1))\n",
    "b_fc1 = tf.Variable(tf.constant(0.1, shape=[1024]))\n",
    "h_pool2_flat = tf.reshape(h_pool2, [-1,7*7*50])\n",
    "h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat,w_fc1) + b_fc1)\n",
    "\n",
    "# dropout（随机权重失活）\n",
    "keep_prob = tf.placeholder(tf.float32)\n",
    "h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)\n",
    "\n",
    "# 设置第二个全连接层\n",
    "w_fc2 = tf.Variable(tf.truncated_normal([1024, 10], stddev=0.1))\n",
    "b_fc2 = tf.Variable(tf.constant(0.1, shape=[10]))\n",
    "y_out =tf.nn.softmax(tf.matmul(h_fc1_drop, w_fc2) + b_fc2)\n",
    "\n",
    "# 建立loss function，为交叉熵\n",
    "loss = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_out), reduction_indices=[1]))\n",
    "# 配置Adam优化器，学习速率为1e-4\n",
    "train_step = tf.train.AdamOptimizer(1e-4).minimize(loss)\n",
    "\n",
    "# 建立正确率计算表达式\n",
    "correct_prediction = tf.equal(tf.argmax(y_out, 1), tf.argmax(y_, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "\n",
    "# 开始喂数据，训练 \n",
    "# with tf.Session() as sess:\n",
    "# 初始化\n",
    "sess.run(tf.global_variables_initializer())\n",
    "# 训练\n",
    "for i in range(20000):\n",
    "    batch = mnist.train.next_batch(50)\n",
    "    if i % 1000 == 0:\n",
    "        train_accuracy = sess.run(accuracy, feed_dict={x:batch[0], y_:batch[1], keep_prob:1})\n",
    "        print(\"step %d,train_accuracy= %g\" % (i, train_accuracy))\n",
    "    sess.run(train_step, feed_dict={x:batch[0], y_:batch[1], keep_prob:0.5})\n",
    "# 算准确率\n",
    "print(\"test_accuracy= %g\" % sess.run(accuracy, feed_dict={x:mnist.test.images, y_:mnist.test.labels, keep_prob:1}))\n",
    "# print('Pred:', y_out.eval(feed_dict={x:mnist.test.images[501].reshape(1,-1), keep_prob:1}))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAADlxJREFUeJzt3X+MVfWZx/HPs9gqmWKiEmFCWWCr4q4mO+jEbMJmgwrVXUmQPyDVZIMsYZqICfgjrvFHaqIkum5dGxOJg0yKkUqJ4xSszZZKzMrqShyVFAsUJoYtyGRGnAasUVF49o85s5ni3O+93HvuPRee9ysh98dzzzlPbvjMOfd+z7lfc3cBiOcvim4AQDEIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoM5p5MbMjNMJgTpzd6vkdTXt+c3sRjP7vZn1mdl9tawLQGNZtef2m9k4SfskzZN0SNI7km5x992JZdjzA3XWiD3/NZL63P1Ddz8uaaOkBTWsD0AD1RL+KZIOjnp8KHvuz5hZh5n1mllvDdsCkLNavvAb69DiG4f17t4pqVPisB9oJrXs+Q9Jmjrq8XclHa6tHQCNUkv435F0qZnNMLNvS/qBpC35tAWg3qo+7Hf3r83sDkm/ljROUpe7/y63zgDUVdVDfVVtjM/8QN015CQfAGcuwg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4KqeopuSTKzA5I+lXRC0tfu3p5HUzg948ePL1mbM2dOctm33norWT969Gg1LTW9Sy65JFnfvn17sj4wMJCsX3fddcn60NBQst4INYU/c627H8lhPQAaiMN+IKhaw++StprZu2bWkUdDABqj1sP+2e5+2MwulvQbM9vr7m+MfkH2R4E/DECTqWnP7+6Hs9tBST2SrhnjNZ3u3s6XgUBzqTr8ZtZiZhNG7kv6vqQP8moMQH3Vctg/SVKPmY2s52fu/p+5dAWg7qoOv7t/KOlvc+wFVXr00UdL1u68887ksvv27UvW58+fn6z39fUl60WaPHlyydrKlSuTy06aNClZT51bIUnLli1L1p944olkvREY6gOCIvxAUIQfCIrwA0ERfiAowg8ElcdVfaiztra2ZH3WrFlVr3vbtm3JejMP5Z177rnJ+vr160vW5s2bV9O2X3/99WT96aefrmn9jcCeHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCYpy/CcyePTtZ7+npSdYnTpxYsrZ3796a1l2kc85J//dcvHhxsl7LWP7bb7+drN9+++3J+hdffFH1thuFPT8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBMU4fwNcf/31yfq6deuS9dQ4viSdPHmyZG3Tpk3JZV977bVkvUhXXXVVsp66Xr+cwcHBZL3c9fj9/f1Vb7tZsOcHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaDM3dMvMOuSNF/SoLtfmT13oaSfS5ou6YCkxe7+x7IbM0tv7Ay1dOnSZP2ZZ55J1sv9/nw5t956a8naxo0ba1p3kbZu3Zqsz507t+p1l/stgJdeeqnqdRfN3a2S11Wy5/+ppBtPee4+Sdvc/VJJ27LHAM4gZcPv7m9IGjrl6QWSRk6vWi/p5pz7AlBn1X7mn+Tu/ZKU3V6cX0sAGqHu5/abWYekjnpvB8DpqXbPP2BmrZKU3Za8SsLdO9293d3bq9wWgDqoNvxbJC3J7i+RtDmfdgA0Stnwm9mLkv5H0kwzO2RmyyQ9Jmmeme2XNC97DOAMUnacP9eNNfE4v1l6aHTmzJkla7t27UouO27cuGQ9dT2+JK1ZsyZZv/vuu0vWjh8/nly2SOXel2PHjiXr48ePT9Y/+uijkrVrr702uWxfX1+y3szyHOcHcBYi/EBQhB8IivADQRF+ICjCDwTFT3dnpk6dmqzv3r27btvevDl9jtSqVauS9RMnTuTZTq5Sw3H33ntv1ctK5YdIX3jhhZK1M3koLy/s+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKC7pzSxfvjxZf/bZZxvUyTd1d3cn65999lnJ2tGjR5PLrly5sqqeKnXDDTeUrJX7eeyWlpZk/f3330/Wr7766mT9bMUlvQCSCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMb5MwcPHkzWp0yZ0qBO8vXll18m611dXcn6ihUrkvWLLrooWe/t7S1ZmzZtWnLZctfrL1q0KFnv6elJ1s9WjPMDSCL8QFCEHwiK8ANBEX4gKMIPBEX4gaDKjvObWZek+ZIG3f3K7LmHJS2X9HH2svvd/VdlN9bE4/y33XZbsr5w4cLGNFKFyy67rGQtNbW4JH3++efJ+iuvvJKs79+/P1l/4IEHkvWU1O8USNKECROqXvfZLM9x/p9KunGM5//D3duyf2WDD6C5lA2/u78haagBvQBooFo+899hZr81sy4zuyC3jgA0RLXhXyPpe5LaJPVL+nGpF5pZh5n1mlnpk7wBNFxV4Xf3AXc/4e4nJa2VdE3itZ3u3u7u7dU2CSB/VYXfzFpHPVwo6YN82gHQKGWn6DazFyXNkTTRzA5J+pGkOWbWJsklHZD0wzr2CKAOuJ7/LDBx4sSStaVLlyaXffzxx5N1s/SQcQXniVS97FNPPZWs33XXXcl6VFzPDyCJ8ANBEX4gKMIPBEX4gaAIPxBU2XF+NL8jR46UrD355JNVLytJDz74YLI+Y8aMZL2WoeShIa4nqyf2/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFJf0Imnu3LnJ+tatW6te91dffZWsn3/++cl6uenHo+KSXgBJhB8IivADQRF+ICjCDwRF+IGgCD8QFNfzB9fa2pqsr127tqb179ixo2TtnnvuSS7LOH59secHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaDKjvOb2VRJz0uaLOmkpE53/4mZXSjp55KmSzogabG7/7F+raIeZs6cmaxPmzatpvXv3bu3ZO3NN9+sad2oTSV7/q8l3e3ufy3p7yStMLO/kXSfpG3ufqmkbdljAGeIsuF39353fy+7/6mkPZKmSFogaX32svWSbq5XkwDyd1qf+c1suqRZknZImuTu/dLwHwhJF+fdHID6qfjcfjP7jqRuSavc/ZhZRT8TJjPrkNRRXXsA6qWiPb+ZfUvDwd/g7i9nTw+YWWtWb5U0ONay7t7p7u3u3p5HwwDyUTb8NryLXydpj7uPnvJ1i6Ql2f0lkjbn3x6AeqnksH+2pH+WtMvMdmbP3S/pMUmbzGyZpD9IWlSfFlGLtra2ZH3Dhg01rf+TTz5J1lNDfShW2fC7+39LKvUB//p82wHQKJzhBwRF+IGgCD8QFOEHgiL8QFCEHwiKn+4+C1x++eUla4888khy2XI/3V3O9u3bk/XnnnuupvWjftjzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQjPOfAa644opkffXq1SVrN910U03bHhgYSNa7u7uT9XLX+6M47PmBoAg/EBThB4Ii/EBQhB8IivADQRF+IChz98ZtzKxxGzuLnHfeecn6xx9/XLLW0tKSXPbVV19N1h966KFkfefOnck6Gs/dK5pLjz0/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRVdpzfzKZKel7SZEknJXW6+0/M7GFJyyWNDDLf7+6/KrMuxvmBOqt0nL+S8LdKanX398xsgqR3Jd0sabGkP7n7v1faFOEH6q/S8Jf9JR9375fUn93/1Mz2SJpSW3sAinZan/nNbLqkWZJ2ZE/dYWa/NbMuM7ugxDIdZtZrZr01dQogVxWf229m35H0X5JWu/vLZjZJ0hFJLukRDX80+Jcy6+CwH6iz3D7zS5KZfUvSLyX92t2fHKM+XdIv3f3KMush/ECd5XZhj5mZpHWS9owOfvZF4IiFkj443SYBFKeSb/v/XtJ2Sbs0PNQnSfdLukVSm4YP+w9I+mH25WBqXez5gTrL9bA/L4QfqD+u5weQRPiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiq7A945uyIpP8d9Xhi9lwzatbemrUvid6qlWdv0yp9YUOv5//Gxs163b29sAYSmrW3Zu1LordqFdUbh/1AUIQfCKro8HcWvP2UZu2tWfuS6K1ahfRW6Gd+AMUpes8PoCCFhN/MbjSz35tZn5ndV0QPpZjZATPbZWY7i55iLJsGbdDMPhj13IVm9hsz25/djjlNWkG9PWxmH2Xv3U4z+6eCeptqZq+b2R4z+52ZrcyeL/S9S/RVyPvW8MN+MxsnaZ+keZIOSXpH0i3uvruhjZRgZgcktbt74WPCZvYPkv4k6fmR2ZDM7N8kDbn7Y9kfzgvc/V+bpLeHdZozN9ept1IzS9+mAt+7PGe8zkMRe/5rJPW5+4fuflzSRkkLCuij6bn7G5KGTnl6gaT12f31Gv7P03AlemsK7t7v7u9l9z+VNDKzdKHvXaKvQhQR/imSDo56fEjNNeW3S9pqZu+aWUfRzYxh0sjMSNntxQX3c6qyMzc30ikzSzfNe1fNjNd5KyL8Y80m0kxDDrPd/SpJ/yhpRXZ4i8qskfQ9DU/j1i/px0U2k80s3S1plbsfK7KX0cboq5D3rYjwH5I0ddTj70o6XEAfY3L3w9ntoKQeDX9MaSYDI5OkZreDBffz/9x9wN1PuPtJSWtV4HuXzSzdLWmDu7+cPV34ezdWX0W9b0WE/x1Jl5rZDDP7tqQfSNpSQB/fYGYt2RcxMrMWSd9X880+vEXSkuz+EkmbC+zlzzTLzM2lZpZWwe9ds814XchJPtlQxlOSxknqcvfVDW9iDGb2Vxre20vDVzz+rMjezOxFSXM0fNXXgKQfSfqFpE2S/lLSHyQtcveGf/FWorc5Os2Zm+vUW6mZpXeowPcuzxmvc+mHM/yAmDjDDwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUP8HwmJTzpKEE6YAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1c05e6ef5f8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample:  4\n",
      "mnist.test.labels:\n",
      " [0. 0. 0. 0. 1. 0. 0. 0. 0. 0.]\n",
      "Pred:\n",
      " [[1.54082378e-12 6.72457035e-10 1.08526825e-11 2.40996784e-12\n",
      "  9.99999881e-01 2.44344683e-11 1.22493704e-13 1.29587596e-09\n",
      "  1.51193504e-07 9.50989357e-11]]\n"
     ]
    }
   ],
   "source": [
    "def showtest(i, sess):\n",
    "    %matplotlib inline\n",
    "\n",
    "    sample = mnist.test.images[i].reshape(28,28)\n",
    "    plt.imshow(sample, cmap='gray')\n",
    "    plt.show()\n",
    "    \n",
    "    print('Sample: ', np.argmax(mnist.test.labels[i]))\n",
    "    print('mnist.test.labels:\\n', mnist.test.labels[i])\n",
    "    print('Pred:\\n', sess.run(y_out, feed_dict={x:mnist.test.images[i].reshape(1,-1), keep_prob:1}))\n",
    "\n",
    "showtest(6205, sess)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
